Skip to content

feat(integrations): unify Tau memory schemas and setup - #1493

Merged
phernandez merged 3 commits into
mainfrom
feat/tau-shared-memory
Sep 7, 2026
Merged

feat(integrations): unify Tau memory schemas and setup#1493
phernandez merged 3 commits into
mainfrom
feat/tau-shared-memory

Conversation

@phernandez

@phernandez phernandez commented Sep 6, 2026

Copy link
Copy Markdown
Member

Why

Follow-up to #1489; refs #1487. Basic Memory's value is shared, structured knowledge that any connected agent can find and understand. Tau's initial integration delivered lifecycle continuity but emitted coding_session notes without the established Git identity fields and used cwd-only recall. Setup also needed schema/placement conventions and a packaged installation path that does not require a source checkout.

This aligns the memory contract without invoking bm hook from Tau, importing CLI orchestration into the extension, or introducing a generic host framework. Sidebar/custom-frontend work remains separate.

What Changed

  • Canonical Coding Session, Session, Task, and Decision schemas in integrations/shared/schemas, with checked copies in Claude Code, Codex, and Tau packages. New checkpoints use the shared agent / session_id pair. Existing agent-specific identity fields remain accepted for old notes; Codex keeps its separate general-session schema.
  • Explicit user-owned coding profiles keyed by absolute checkout root, with confirmed repository identity, their own write project, read-only recall sources, and placement conventions. Coding session notes default to {chosen project}/tau/{repo name}, using stable repository identity rather than the worktree basename. General sessions retain tau/checkpoints; transcripts retain tau/transcripts; explicit folder overrides win.
  • Coding checkpoints carry actual Git root/branch/SHA, optional PR fields, project/timestamps/capture method, and shared session identity. General checkpoints use session. Synthesis uses shared schema categories.
  • Repository-scoped coding recall across worktrees, active tasks/open decisions, approved read-only shared sources, and bounded knowledge discovery. No unscoped recent-session feed in coding profiles; unconfigured nested Git checkouts cannot inherit their parent's coding route.
  • bm install tau --dry-run and bm install tau --sync: packaged extension, setup/shared skills, and four prompt templates. The installer previews and confirms changes, preserves existing configuration/credentials, reuses shared resources, refuses conflicting copies and unapproved replacements, and optionally syncs an isolated pinned Tau environment.
  • A named Basic Memory extension entry point, plus /bm-resume, /bm-plan, /bm-decide, and /bm-wrap-up templates.
  • Guided setup with destination/profile consent, cost/privacy disclosure, approved schema seeding, config/schema preservation, separate connectivity/write verification, and reload/shutdown caveats.
  • Pins the Basic Machines Tau fork at d8216af0b34059839734422d1e39b4c5972d7966, including the single-copy snapshot fix. Upstream contribution: feat: support awaited extension continuity across compaction huggingface/tau#687.

Implementation Details

knowledge.py owns validated profiles, bounded asynchronous Git metadata reads, and session placement. Coding placement takes the final component of the user-confirmed repository identity; multiple explicitly registered worktrees therefore share a folder. An omitted/null checkpoint_folder selects automatic placement. Existing string overrides remain authoritative, including tau/checkpoints.

The lifecycle stays in continuity.py; MCP ownership, argument forwarding, and receipt identity retain their existing boundaries. Missing/failed/timed-out optional GitHub lookup omits PR context; invalid successful payloads and required Git failures surface rather than inventing identity. Secondary projects participate only in automatic reads. Explicit agent tool arguments remain unchanged.

Hook-backed Claude Code/Codex writers and deliberate checkpoint skills now emit the shared identity pair as well. Legacy note fields remain readable, and same-chat lineage must match host plus session ID rather than a bare ID.

The wheel explicitly includes Tau resources, excluding integration environments/caches. Installation locates distribution-owned resources (including editable-install resource behavior), bypasses normal CLI initialization, and does not read/change user config, initialize the database, or write notes. --sync is separate opt-in dependency installation; it does not change global Tau. Launch with uv run --project ~/.tau/extensions/basic-memory tau without a second explicit -e load.

Canonical schemas are bundled as self-contained copies checked by scripts/sync_memory_schemas.py --check. Existing user schemas, notes, and durable receipts are not rewritten automatically.

Testing

Latest local verification before pushing:

  • just fast-check: passed, including root Ruff, format, and just typecheck.
  • uv run pytest tests/cli/test_install_tau.py tests/cli/test_install_tau_wheel.py tests/cli/test_coding_session_context.py tests/cli/test_hook_command.py tests/test_codex_plugin_package.py -q: 159 passed, 1 opt-in wheel test skipped.
  • just package-check: passed across all packages. Tau: 76 passed, 4 opt-in real-BM tests skipped, Ruff/format/ty passed.
  • BM_TAU_TEST_COMMAND="$PWD/.venv/bin/bm" uv run --project integrations/tau --with pytest-cov pytest -c integrations/tau/pyproject.toml integrations/tau/tests --cov=tau --cov-config=integrations/tau/pyproject.toml --cov-report=term-missing -q: 80 passed; 100% statement coverage across six Tau runtime modules (623 statements). An initial invocation without Tau-local coverage configuration failed before collection because the root coverage configuration requires gevent; the corrected command above passed.
  • uv build --wheel --out-dir /tmp/tau-pr-wheel, followed by BM_TAU_INSTALL_WHEEL=<built-wheel> uv run pytest tests/cli/test_install_tau_wheel.py --no-cov -q: 1 passed. Exercises preview, installation, and repeat installation from an extracted wheel outside the checkout, without creating config/database state.
  • just doctor: passed in a temporary project.
  • git diff --check: passed.

Tests cover coding/general default placement, explicit overrides and configuration round trips, actual checkpoint write destinations, Git/PR metadata and cancellation, nested/unborn checkout safety, read-only source routing, real Tau resource loading/expansion, installer approval/conflict behavior, and schema parity. Real-BM tests validate Tau checkpoints and cross-worktree recall of both shared-identity and legacy other-host notes. Existing compaction/reload/branch/resume, TUI reload, receipt reconciliation, privacy, and persistent MCP tests pass.

The user reports the pinned installed launch works. Automated model synthesis uses deterministic fake providers; no paid live-model, production-memory, or cloud-account test was performed by the assistant for this update. Full SQLite/Postgres suites were not rerun.

Risks / Follow-ups

  • Requires upstream Tau FTS-based session/conversation search for episodic recall #687 or the pinned fork. No installed Tau packages or the user's installed extension/configuration were patched in this update.
  • Updating an installed copy requires reinstalling packaged resources; existing explicit folder overrides are preserved, and old notes are not moved.
  • Automatic synthesis adds model cost/latency; recalled content reaches the active model. Captures use the explicitly selected destination; credential masking remains best-effort.
  • Reload shuts down the old lifecycle first, so its final capture can still use previous settings.
  • Custom schema migrations require explicit approval. Confirmed captures retain receipt identity and are reused rather than rewritten because a profile/schema changed.
  • Repository names without owner namespaces can share a folder across owners; recall remains scoped by complete repository identity.
  • Destination/recall/checkpoint sidebar work is a separate follow-up.

No merge is authorized.

Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-07T17:28:57.097367Z dfbff24 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@phernandez

Copy link
Copy Markdown
Member Author

Manual acceptance test plan

Tester: Paul. Status: not yet run. Automated results above do not substitute for this live-model/interactive pass. Record pass/fail and the returned note paths below. Use non-sensitive test content and a disposable local Basic Memory project. Model synthesis incurs normal provider charges. Do not use a team/production destination for this pass.

1. Launch safely from the branch

Use the existing checkout below, or substitute your checkout of feat/tau-shared-memory. Run these in a shell, not as a Tau prompt:

export BM_SRC=/Users/phernandez/dev/basicmachines/basic-memory-tau-1487
# Confirm feat/tau-shared-memory and the current PR head before testing.
git -C "$BM_SRC" branch --show-current
git -C "$BM_SRC" rev-parse HEAD
uv sync --project "$BM_SRC/integrations/tau"

export BM_TEST_ROOT="$(mktemp -d)"
export TAU_BASIC_MEMORY_CONFIG="$BM_TEST_ROOT/basic-memory.json"
cat > "$TAU_BASIC_MEMORY_CONFIG" <<'JSON'
{
  "project": null,
  "auto_recall": false,
  "capture_knowledge": false,
  "checkpoint_on_compact": false,
  "summarize_on_shutdown": false,
  "capture_transcript": false
}
JSON
chmod 600 "$TAU_BASIC_MEMORY_CONFIG"
mkdir "$BM_TEST_ROOT/repo"
git -C "$BM_TEST_ROOT/repo" init -b main
git -C "$BM_TEST_ROOT/repo" -c user.name='BM Test' \
  -c user.email='bm-test@example.invalid' -c commit.gpgsign=false \
  commit --allow-empty -m 'Initialize disposable memory test'
cd "$BM_TEST_ROOT/repo"
uv run --project "$BM_SRC/integrations/tau" tau -e "$BM_SRC/integrations/tau"

The launch uses the pinned organization fork while preserving the test repo as cwd. Do not launch from the BM source directory when testing this repo's profile. Keep the same shell exports for subsequent launches. The separate config does not isolate provider credentials or Tau's normal session storage; use test conversations only. Do not load a second installed copy of the BM extension.

  • /bm-status reports connected, no write destination, and all automatic controls off. No automatic notes are created.
  • Normal chat and explicit BM read tools work. A configured model/provider is required for chat; do not change credentials as part of this test.

2. Test the setup interview

The skill need not be installed globally. In Tau, ask it to read the absolute file path shown by BM_SRC above, ending in:

integrations/tau/skills/basic-memory-setup/SKILL.md

Suggested prompt:

Read that setup skill and walk me through configuring this disposable checkout. Use the existing TAU_BASIC_MEMORY_CONFIG override, not ~/.tau/basic-memory.json. Ask before choosing the project or writing schemas. I want a coding profile and full continuity, with transcripts off. This test repo has no remote: ask me to confirm the test identity rather than inventing one.

Choose an existing disposable local BM project, or explicitly approve creating one as a separate step. For the test-only repository label, confirm bm-test/continuity. Confirm the actual Git root and retain project: null at the general level so unregistered directories remain tools-only.

  • The assistant asks for the destination, coding/general choice, repository identity, capture policy, optional read sources, and schema writes. It explains model costs and privacy.
  • It edits only the test config. The repository entry contains the actual absolute test root, kind: coding, repository: bm-test/continuity, and the selected project. Other config values are preserved.
  • It offers Coding Session, Task, and Decision schemas. Approved missing schemas are written once; rerunning setup does not overwrite existing/customized schemas.
  • After /reload, /bm-status shows the selected project, coding profile, expected read sources, and full-continuity controls. No extra assistant turn is triggered just by reload.

3. Verify structured knowledge, not just a saved summary

Send a small test task:

Our test decision is to paint the astrolabe blue rather than red because blue matches the existing instrument labels. We have not painted it or run any tests. Next action: inspect the paint samples. Preserve that distinction in memory.

Wait for the turn to settle, then inspect the confirmed checkpoint using BM tools or your note viewer.

  • /bm-status has a confirmed checkpoint path and no failure. The file exists in the selected project's checkpoint folder.
  • Frontmatter has type: coding_session, the selected project, repository: bm-test/continuity, actual repo_root/cwd, branch, SHA, timestamps, tau_session_id, and capture: summarized.
  • Git values match git rev-parse --show-toplevel, git branch --show-current, and git rev-parse HEAD in the test checkout. With no PR, it does not invent PR metadata.
  • The note has categorized observations and a clear next step. It does not claim the paint work or tests were completed.
  • Ask Tau to validate that exact note with the advertised bm_schema_validate tool. Expect no required-field warnings/errors after the shared schema is seeded. If the tool is unavailable, record that stage as unverified rather than enabling server features silently.
  • Ask for a search using note_types=["coding_session"] and metadata_filters={"repository":"bm-test/continuity"} in the selected project. The checkpoint is found without a Tau-specific receipt lookup.
  • Try /bm-remember Record the astrolabe color decision with its rationale in our decision folder. Verify a deliberate decision note with its observation/rationale and expected placement, rather than another checkpoint-shaped dump. Read its returned path; do not accept an acknowledgement as proof of a save.

4. Reload, checkpoints, compaction, and shutdown

  • Immediately after a settled turn, run /bm-checkpoint twice without another ordinary chat turn between them. Expect the same confirmed source-tip checkpoint, not repeated copies. Inspect note counts outside the chat or through the note viewer so inspection does not itself create a new public turn.
  • /reload, then /bm-orient astrolabe. The prior work is inserted as historical reference, with the correct write destination and placement guidance. Neither command should cause an unsolicited extra assistant response.
  • In this disposable session, run /compact after several turns. If Tau reports no compaction needed, record that and retry after more context; a no-op does not prove checkpoint restoration. On actual compaction, verify a confirmed checkpoint reference is restored. Then ask what remains unfinished: it should recover the paint-sample step without claiming completion.
  • For a shutdown-specific check, temporarily set capture_knowledge: false while leaving summarize_on_shutdown: true, reload, add one new test fact, then quit normally. Read the final checkpoint after exit and verify that outstanding fact was saved. Restore the intended controls before other tests. A process kill is not a shutdown test.
  • Resume the test session and verify its prior checkpoint can be recovered. No ambiguous or failed save should ever be presented as confirmed.

5. Cross-worktree and repository isolation

Exit the test Tau session normally first. From the same shell:

git -C "$BM_TEST_ROOT/repo" worktree add -b second-checkout "$BM_TEST_ROOT/worktree"

With approval, add a second repository entry to the test config, copying the explicitly approved project/read-source/placement choices, using root $BM_TEST_ROOT/worktree and the same repository: bm-test/continuity. Keep the first entry intact. Start a new Tau session from the second worktree using the absolute launch command in section 1.

  • Startup recall or /bm-orient astrolabe finds the first checkout's coding checkpoint despite a different cwd and no inherited Tau receipts.
  • In another disposable Git repo, configure a different identity, e.g. bm-test/unrelated, using the same BM project. Its coding-session recall must not present the astrolabe checkpoint as this repository's work. Project-wide tasks/decisions can still appear as shared project knowledge; they are not repository-isolated by this feature.
  • Optional safety case: make an unregistered nested Git repo beneath the first root and launch from it. Expect a visible profile/root mismatch failure and no parent-repository checkpoint or transcript writes, not a silent fallback.

6. Optional read-only source and general-profile checks

  • Choose a second disposable project containing a known open decision or active task. Explicitly approve it as a read_projects source and run /bm-orient. Its note content appears with read-only source labeling. Automatic checkpoints still go only to the primary project. Do not ask the agent to write to the secondary project during this check: explicit tool calls are not a read-only security boundary.
  • For general use, work outside every configured coding root and explicitly choose a disposable general project in the test config. Create a short test conversation. Expect a session checkpoint, not coding_session, and no invented Git metadata. Seed/validate against the Session schema if testing this stage.

7. Finish and report

Quit test sessions normally before changing destinations or disabling capture: the old lifecycle can still write during shutdown/reload using its previous settings. Do not delete a worktree while its Tau session is running.

  • Report the tested head, completed sections, any failures, relevant note paths, and whether live synthesis quality was acceptable. Redact secrets and private project content from screenshots/logs.
  • Retain test artifacts until results are understood. Remove test notes, schema seeds, worktrees, or config only deliberately; no automatic cleanup of checkpoints/receipts. unset TAU_BASIC_MEMORY_CONFIG in the test shell restores the ordinary config choice for future launches; it does not alter an already-running Tau session.

Manual result: pending tester execution.

Signed-off-by: phernandez <paul@basicmachines.co>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfbff24f1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

prior = checkpoint_records[-1] if checkpoint_records else None
tip = entries[-1].id
capture_id = digest([self.settings.project, context.session_id, "checkpoint", tip])
capture_id = digest([self.profile.project, context.session_id, "checkpoint", tip])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Scope capture identities to the repository

When a persisted Tau session is resumed in another configured repository that writes to the same Basic Memory project, both context.session_id and the source entry ID remain unchanged, so this hash collides with the prior repository's capture. The subsequent prior/recover paths can then return that old checkpoint as confirmed without validating its repository, leaving the current repository without a checkpoint and potentially pointing compaction at unrelated work. Include the coding profile's repository identity in checkpoint and transcript capture IDs, and apply the same scope when recovering captures.

Useful? React with 👍 / 👎.

@phernandez
phernandez merged commit b7b8f39 into main Sep 7, 2026
43 checks passed
@phernandez
phernandez deleted the feat/tau-shared-memory branch September 7, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant